1. Technical Field
The invention is related to adding high-frame rate data to a low frame rate video stream, and more particularly to highlighting the current speaker in each frame of a low frame-rate video.
2. Background Art
Meetings are an important part of everyday life for many workgroups. A variety of live teleconferencing systems are available commercially to enhance and archive these meetings. Both recorded meetings and live video conferences require audio/visual (A/V) capturing equipment. One such system generally referred to as a Distributed Meeting (DM) system provides high quality two-way conferencing and recording of meetings, as well as rich browsing of archived meetings enhanced through a number of analysis techniques. DM systems typically have multiple cameras that simultaneously capture different sub-events occurring in a space where an event occurs, such as a meeting room, to give a rich experience to local and remote meeting participants. These capture devices are all synchronized to provide an integrated, scalable system and method for two-way conferencing, broadcasting, recording and viewing meetings or other events. In particular, a DM system's cameras can include a 360-degree camera centrally positioned to monitor in 360 degrees the space in which the event occurs; a remote view camera positioned so as to capture a view of event participants in the meeting room; a presenter view camera positioned so as to capture a view of the front of the meeting room where a presenter is typically presenting; and a whiteboard capture camera positioned so as to capture strokes written on a whiteboard. A DM system also typically has one or more microphone devices or microphone arrays that simultaneously capture audio data that is synchronized with the image data recorded by the various cameras. A given microphone device or array can be used to determine the direction from which an event participant is speaking by using sound source localization. The 360 degree camera, presenter view camera and remote view camera all can have microphone arrays associated with them. In addition, at least one server or other suitably configured computer that is capable of storing and distributing the captured data to one or more clients in network connection with the server can be included. For live meetings using a DM system, the system broadcasts the multimedia meeting streams to participants, who use a network for low-latency duplex voice and data communication. The meetings can also be recorded and viewed on-demand.
Even with videoconferencing systems such as the DM system available, many businesses do not take advantage of the benefits these systems have to offer. Often the most important impediment for deploying such videoconferencing systems is the bandwidth consumption, which may reach hundreds of kilobits per second for each video stream. For example, sending a complete panoramic image from the 360 degree camera of a DM system with at a full video frame rate would be catastrophic from the bandwidth usage point of view, even with state-of-the-art video compression like WMV9 and H.264. In many applications, however, the importance of a high frame-rate video is of secondary importance compared to the audio portion. It is not necessary for remote participants to see even motion an attendee makes. It is however important for a remote participant to know who is in the meeting and talking so as to better follow the conversation. Thus, the quality of the teleconferencing experience can be significantly improved, even when bandwidth is at a premium, by providing video frames at a slower rate, but with an ongoing indication of the current speaker. The present invention provides such a capability with a low bandwidth cost for use with the foregoing bandwidth limited videoconferencing applications, and more generally for any event where multiple people are attending and it is important to know who is currently talking.
It is noted that in the remainder of this specification the description refers to various individual publications identified by a numeric designator contained within a pair of brackets. For example, such a reference may be identified by reciting, “reference [1]” or simply “[1]”. Multiple references will be identified by a pair of brackets containing more than one designator, for example, [2, 3]. A listing of references including the publications corresponding to each designator can be found at the end of the Detailed Description section.